Design Principles for a Robust Network Infrastructure

The Internet is now central to a huge amount of our economy, and yet we are unable to depend on its continued and smooth operation. For some kinds of failures, the Internet has proven to be extremely robust. It has survived hurricanes, earthquakes, tunnel fires, and terrorist attacks with only temporary and partial loss of end-to-end connectivity. At the same time, other, seemingly trivial failures are able to disrupt the Internet largely without bound. This is because a fundamental assumption underlying the design of many Internet protocols is that systems are fail-stop – they completely and detectably stop working when they fail. Not all failures are so clean. Implementation bugs, configuration mistakes and malicious attacks may cause systems to obey the syntax of a protocol but in fact be behaving incorrectly, in ways that undermine the Internet’s operation. These arbitrary failures occur with a surprising regularly and, unlike simple failures, can cascade through the system with serious consequences to parties not at fault. In one recent example, widespread outages were triggered because one vendor’s BGP implementation ignores but propagates incorrect route announcements, while another vendor’s routers terminate any BGP session propagating an obviously incorrect announcement. Over the past few years, operator errors in the form of router configuration mistakes have led to several spectacular disruptions to Internet connectivity. As a further example, the CodeRed and Nimda worms triggered previously unknown BGP instabilities. To put these incidents in perspective, consider that each of them arguably caused a more serious Internet outage than the September 11 terrorist attacks in which a vast amount of connectivity through New York City was severed by the collapse of the World Trade Center.